Image creation using geo-fence data

ABSTRACT

An unmanned aerial vehicle (UAV) may perform a surveillance action at a property of an authorized party. The property may be defined by a geo-fence, which may be a virtual perimeter or boundary around a real-world geographic area. The UAV may image the property to generate surveillance images, and the surveillance images may include image data of objects inside the geo-fence and image data of objects outside the geo-fence. While gathering surveillance images, or after the surveillance images have been gathered, the geo-fence information may be used to obscure or remove image data referring to objects outside the geo-fence. Geo-clipped surveillance images may be generated by physically constraining a sensor of the UAV, by performing pre-image capture processing, or post-image capture processing. Geo-clipped surveillance images may be limited to authorized property, so privacy is ensured for private persons and property.

BACKGROUND

Traditional home surveillance may include video cameras installed by aservice provider to monitor a property. The viewpoint of these videocameras may be fixed or may have limited movable range and therefore thevideo cameras may miss important events. Additionally, with the largeamount of video that can be captured, it is possible to miss importantevents. Furthermore, traditional ground-based surveillance systems maybe vulnerable to manipulation or damage, for example, by an intruder.For example, an intruder may disable or break a ground-based videocamera when the video camera is at a known location.

At the same time, as the delivery of packages using unmanned aerialvehicles (UAVs) becomes prevalent, UAVs traveling to and from a deliverydestination may be leveraged to perform secondary tasks. With a varietyof sensors aboard, including a digital camera, a UAV may be deployed toperform secondary tasks that are different than delivering a package toa destination and then returning directly back to an originationlocation.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Thesame reference numbers in different figures indicate similar oridentical items.

FIG. 1 is a schematic diagram showing an illustrative environment wherean unmanned aerial vehicle may deliver a package as a primary task andprovide surveillance as a service as a secondary task.

FIG. 2 is a block diagram of an example surveillance system.

FIG. 3 is a flow diagram of an example process for performingsurveillance while delivering a package with an unmanned aerial vehicle.

FIG. 4 is a flow diagram of an example process for performingsurveillance while delivering a package with an unmanned aerial vehicle.

FIG. 5A is a schematic diagram illustrating a surveillance image,processing of the surveillance image, and a post-processing surveillanceimage, in accordance with one embodiment disclosed herein.

FIG. 5B is a schematic diagram illustrating a post-processingsurveillance image, surveillance events, and surveillance alerts, inaccordance with one embodiment disclosed herein.

FIG. 6 is a flow diagram of an example process for processingsurveillance data.

FIG. 7A is a flow diagram of an example process for processing aninterrupt to perform a surveillance action.

FIG. 7B is a flow diagram of an example process for selecting asurveillance action.

FIG. 8 is an illustrative user interface associated with determiningparameters of a surveillance system.

FIG. 9 is a block diagram of components of an example unmanned aerialvehicle that is supported by the surveillance system.

DETAILED DESCRIPTION

This disclosure provides methods and systems for using an unmannedaerial vehicle (UAV) to perform a surveillance action of personalproperty that is verified as belonging to a user. For example, a UAV mayhave a primary role of delivering a package to a user. In the course ofdelivering the package, the UAV may determine whether it has availableresources, and if so, the UAV may perform an additional scheduledsurveillance action. In one embodiment, a surveillance action mayinclude flying over a house of a different user who has consented tosurveillance and gathering surveillance data, such as by recording videoof the user's property while excluding adjacent properties (possibly bypost-capture editing). After surveillance data has been gathered, thedata may be analyzed to determine if there is a surveillance event. Anexample of a surveillance event may be the determination that a garagedoor was left open during the day, a broken window, a detection ofgraffiti, or a fire. In one embodiment, after a surveillance event hasbeen determined, an alert may be provided to a user or a serviceprovider.

In various embodiments, a user may subscribe to a surveillance system toprovide surveillance as a service. The user may provide variousparameters for the surveillance, such as a surveillance tier, frequency,monitoring type (e.g., sensors), and alerts. In some embodiments, a usermay specify an object of interest for the surveillance system.

In various embodiments, surveillance data gathered by the UAV may bemodified to present a geo-clipped image or video to a user. Ageo-clipped image or video may be an image or video that has beenmodified by at least one location-based parameter. For example, ageo-clipped image or video may mask an image to show only thoseconsented locations owned or controlled by a user. Such a geo-clippedimage may ensure privacy of neighbors, for example. To access such aservice, a user may have to submit proof of ownership or control of thelocation receiving surveillance, and possibly consent of other peopleresiding at that location.

In various embodiments, a surveillance system may propose possiblesurveillance actions, and may include the proposed surveillance actionsin a flight plan for a UAV. During the course of delivering a package,the UAV may evaluate its resources and select a surveillance actionbased on the UAV's available resources and a priority level of thesurveillance action.

The techniques and systems described herein may be implemented in anumber of ways. Example implementations are provided below withreference to the following figures.

FIG. 1 is a schematic diagram showing an illustrative environment 100where an unmanned aerial vehicle (UAV) may deliver a package as aprimary task and provide surveillance as a service as a secondary task.The environment 100 may correspond to a delivery area where UAVs deliverinventory items to various destination locations. The UAV 104 may bedispatched to deliver a package 106 to a destination in the environment100. The UAV 104 may begin its delivery mission from an originationlocation 102, such as a fulfillment center where inventory is stored,and may fly along a flight path 108 to a destination location 110. Insome embodiments, the flight path 108 may be considered to be anoutbound leg to deliver the package 106. The UAV 104 may continue on toanother destination location if it is carrying multiple items that canbe individually delivered to multiple destination locations.

After the package 106 has been delivered to the destination location110, the UAV 104 may travel along a flight path 112 to return to theorigination location 102. In some embodiments, the flight path 112 maybe considered to be an inbound leg used to efficiently return the UAV104 to the origination location 102. In some embodiments, the UAV 104may divert from the flight path 112 and begin a new flight path 114,which may be referred to as a surveillance leg. Any number of events maytrigger a diversion from a trip, such as a surveillance interrupt, or adetermination that remaining resources of the UAV 104 are sufficient toconduct an approved surveillance action. Events triggering a diversionfrom a trip will be explained in connection with various figures of thisdisclosure. The surveillance action may include a minor deviation, or nodeviation at all from the flight path 112 to return to the originationlocation 102.

The flight path 114 may direct the UAV 104 over one or more excludedlocations 116 en-route to a surveillance location 118. The excludedlocation 116 may be adjacent to a surveillance location and may be alocation that is not authorized or approved to receive surveillance,such as a location where owners, occupants, local ordinances, laws, orother people have not consented to or authorized surveillance.Alternatively, the excluded location 116 may be the subject of aseparate surveillance action; however, relative to surveillance of thelocation 118, the location 116 may be an excluded location.

The surveillance location 118 may be defined by a geo-fence 120. Ageo-fence may create a virtual perimeter or boundary around a real-worldgeographic area, such as area 120. The geo-fence 120 may correspond to aproperty line associated with the surveillance location 118, forexample. A geo-fence may be pre-defined or may be dynamically defined.That is to say, the geo-fence 120 may be generated prior to asurveillance action or may be defined by property records, for example,or may be generated in real time during the surveillance action or anysubsequent analysis.

When the UAV 104 arrives at the surveillance location 118, the UAV mayperform a surveillance action, such as imaging the surveillance location118. As will be described in reference to the various figures in thisdisclosure, a surveillance action may include still imaging, videocapture, use of spectral cameras, audio surveillance, chemicalsurveillance, and/or other types of surveillance.

After the surveillance action is performed at the surveillance location118 to gather surveillance data, the surveillance data may be analyzedand/or modified prior to access by a user. For example, in manyinstances, it may be difficult to gather surveillance data of thesurveillance location 118 without inadvertently gathering surveillancedata of excluded location 116, and it may be necessary to modify thedata to remove data of the excluded location 116 from the surveillancedata to protect the privacy of locations and people not associated withthe service. In the illustrated embodiment, the surveillance data ismodified to generate a geo-clipped image 122. The geo-clipped image 122includes an authorized section 124 and an unauthorized section 126,which is masked or otherwise manipulated such that a user cannot viewimagery of the unauthorized section 126. The unauthorized section 126corresponds to a geo-clipped section that is not subject tosurveillance, such as the excluded location 116, for example. In theillustrated embodiment, the authorized section 124 includes thesurveillance location 118, as well as the surrounding area within thegeo-fence 120.

After a surveillance action has been performed, the UAV 104 may receiveor determine a new flight path, such as flight path 128, and may returnto the origination location 102. Additionally, the UAV 104 may continueon to another destination location and/or another surveillance location.

FIG. 2 illustrates an example central controller 202. In variousexamples, central controller 202 can provide the exemplary processdescribed with respect to FIG. 1. The central controller 202 may includeone or more processor(s) 204 that interact with a computer-readablemedia 206. The computer-readable media 206 may include an operatingsystem 208 and a data store 210 to store delivery data, surveillancedata, scheduling data, or sensor data received from an UAV. In variousembodiments, the data store 210 may store data received from the UAV104. The computer-readable media 206 may also include software programsor other executable modules that may executed by the one or moreprocessor(s) 204. Examples of such programs or modules include, but arenot limited to, scheduling algorithms, surveillance algorithms, sensoralgorithms, data analysis algorithms, network connection software, andcontrol modules.

Various instructions, methods, and techniques described herein may beconsidered in the general context of computer-executable instructions,such as program modules, executed by one or more computers or otherdevices. Generally, program modules include routines, programs, objects,components, data structures, etc. for performing particular tasks orimplementing particular abstract data types. These program modules canbe implemented as software modules that execute on the processing unit,as hardware, and/or as firmware. Typically, the functionality of theprogram modules may be combined or distributed as desired in variousembodiments. An implementation of these modules and techniques may bestored on or transmitted across some form of computer-readable media.

In various embodiments, the computer-readable media 206 may include adelivery scheduling module 212. In various examples, the deliveryscheduling module 212 may receive orders and schedule deliveries by aUAV to a destination location. The delivery scheduling module 212 mayinclude a queue that receives orders and fulfills orders in the sequencethey were received. In other examples, the delivery scheduling module212 may fulfill an order based on a priority of the order, whether apackage is ready to be delivered, weather conditions, or based on anavailability of a UAV. In other examples, orders may be grouped togetherand scheduled to minimize total flight time.

In some embodiments, the delivery scheduling module 212 may determinedelivery requirements, determine the amount of UAV resources to be usedby the delivery, and determine an anticipated unused capacity of theUAV. In some embodiments, the delivery scheduling module 212 maydetermine delivery requirements and an anticipated unused capacity of anUAV for each order. Some of the described embodiments of the deliveryscheduling module 212 are also discussed in connection with FIG. 7B andother figures of this disclosure.

In some embodiments, the delivery scheduling module 212 may determinethat a particular delivery requires more resources than are available toa UAV. In this case, a delivery may not be scheduled, or may berescheduled at another time.

In other examples, the delivery scheduling module 212 may schedule adelivery based at least in part on the surveillance scheduling module214. For example, if a high priority surveillance action is to beperformed, a delivery may be scheduled by the delivery scheduling module212 to preserve available resources for the surveillance action.

In various embodiments, the computer-readable media 206 may include asurveillance scheduling module 214. In various examples, thesurveillance scheduling module 214 may schedule an approved surveillanceaction based in part on the anticipated unused capacity determined bythe delivery scheduling module 212. The surveillance scheduling module214 may receive a request for a surveillance action by an authorizeduser, determine an anticipated amount of UAV resources required by thesurveillance action (e.g., resource requirements such as a powerrequirement, a time requirement, or a sensor requirement), and schedulethe surveillance action with a delivery action scheduled by the deliveryscheduling module 212. The surveillance action is scheduled so that thesurveillance action may be completed without exhausting the remainingresources of the UAV. In various embodiments, the surveillancescheduling module 214 includes a queue that receives multiple requestsfor surveillance actions, and schedules the surveillance actions basedon a priority, the order the requests were received, the amount ofanticipated resources required by the surveillance actions, and/or anamount of anticipated unused capacity determined by the deliveryscheduling module 212.

In some embodiments, the surveillance scheduling module 214 may schedulemultiple surveillance actions with a single delivery action. In someembodiments, the surveillance scheduling module 214 may schedule asurveillance action without performing a delivery scheduled by thedelivery scheduling module 212.

In various embodiments, the computer-readable media 206 may include adata analysis module 216. In various examples, the data analysis module216 may receive surveillance data and modify the surveillance data togenerate geo-clipped surveillance data. For example, the geo-clippedsurveillance data may be a geo-clipped still image or video, such as thegeo-clipped image 122 of FIG. 1, or geo-clipped image 516 of FIG. 5A. Insome embodiments, the data analysis module 216 may receive surveillancedata, register surveillance data, and compare surveillance data with theregistered surveillance data. In some embodiments, the data analysismodule 216 may receive surveillance data and analyze the data todetermine a surveillance event. Some of the described embodiments of thedata analysis module 216 are also discussed in connection with FIG. 3and other figures of this disclosure.

In various embodiments, the computer-readable media 206 may include asurveillance subscription module 218. In various examples, surveillancesubscription module 218 may include a user interface such as userinterface 800 in FIG. 8. Surveillance subscription module 218 may setsurveillance parameters such as a surveillance tier, frequency,monitoring type, objects of interest, and alerts. In some embodiments,surveillance subscription module may determine subscription pricing, andaccept payment information. As discussed in connection with this andother embodiments, the surveillance subscription module 218 may only beused to conduct surveillance of locations where a user is authorized toperform surveillance. For example, a user may have to submit proof ofownership or control of the location receiving surveillance, andpossibly consent of other people residing at that location (e.g., in anapartment building). In this manner, the privacy of locations andpersons is ensured.

In various embodiments, the scheduling system 202 includes one or morecommunication interfaces 220 for exchanging messages with an UAV, asurveillance location, a service provider, various user devices, andother networked devices. The communication interfaces 220 can includeone or more network interface controllers (NICs), I/O interfaces, orother types of transceiver devices to send and receive communicationsover a network. For simplicity, other components are omitted from theillustrated device. In at least one embodiment, the communicationinterfaces 220 receive sensor data, including surveillance data, fromthe UAV.

FIG. 3 is a flow diagram of an example process 300 for performing anapproved surveillance while delivering a package with an unmanned aerialvehicle. In some embodiments, the process 300 may be performed by thecentral controller 202, the UAV 104, or both. Some of the operations inthe process 300 may be performed in parallel or possibly in a differentorder than the order shown in FIG. 3.

At 302, a UAV may be in transit to or from a delivery. For example, thedelivery may be a delivery of the package 106 to the destinationlocation 110, as shown in FIG. 1. The example process 300 in FIG. 3 maybe performed before or after a delivery has been performed, or may beperformed between deliveries, if the UAV is capable of making multipledeliveries in a single trip.

At 304, the resources of the UAV may be determined. In some embodiments,resources of the UAV include a power resource, a sensor resource, or atime resource. For example, a power resource may indicate the remainingbattery power or fuel levels for the UAV, and may also indicate anavailable range of the UAV. A sensor resource may indicate the status ofthe available sensors on the UAV, such as a digital camera, an availableamount of memory for gathering surveillance data, or a communicationcapability. A time resource may indicate a time constraint on the UAV.For example, the UAV may be scheduled to deliver a package by apredetermined time, or the UAV may be scheduled to depart for its nextdelivery at a predetermined time. In some embodiments, the time resourcemay reflect a demand for a UAV at a fulfillment center. Determining theresources of the UAV may also include determining the resources requiredby a delivery action, and/or determining the resources required by asurveillance action. The resources to be determined may be resourcesused or resources anticipated to be used. In some embodiments, thedetermination is based in part on environmental factors such as theweather, including wind speed, direction, a weather forecast,temperature, time, etc. In some embodiments, determining the resourcesmay also include determining a priority of a resource, a deliveryaction, or a surveillance action.

At 306, the UAV may deviate from transit. Included with deviating fromtransit, operation 306 may also include a determination that sufficientresources are available for a surveillance action. Deviating fromtransit may include computing a new flight path or may include selectinga previously computed flight path. Deviating from transit may alsoinclude selecting a surveillance action from one or more possiblesurveillance actions. Deviating may include changing speed, heading,and/or altitude. An example of deviating from transit is shown as theflight path 114 in FIG. 1.

At 308, the UAV may perform a surveillance action. In some embodiments,a surveillance action includes taking still images or video of asurveillance location, such as the surveillance location 110. Variousexamples of surveillance actions include using any sensors 904 of UAV902, such as digital camera(s) 906, spectral camera(s) 908, audiosensor(s) 910, LIDAR/RADAR 912, GPS sensor(s) 914, chemical sensor(s)916, flight/delivery sensor(s) 918, or any combination thereof.Furthermore, surveillance actions are described in detail in connectionwith the various figures in this disclosure.

At 310, the UAV may resume transit, which may include continuing with adelivery, aborting a delivery, returning to an origination location ofthe UAV, or traveling to another location. Operation 310 may furtherinclude determining resources remaining after performing a surveillanceaction, in order to determine a flight path to resume.

FIG. 4 is a flow diagram of an example process 400 for performingsurveillance while delivering a package with an unmanned aerial vehicle.

In some embodiments, operations 402-412 may be performed by a UAV, suchas the UAV 104, while operations 414-426 may be performed by the centralcontroller 202. In some embodiments, the UAV 104 may perform alloperations, or the UAV 104 and the central controller 202 may performoperations 402-426 in any combination, as would be understood by aperson of ordinary skill in the art. Some of the operations in theprocess 400 may be performed in parallel or possibly in a differentorder than the order shown in FIG. 4.

At 402, the UAV 104 delivers a package at a destination location. Insome embodiments, the UAV 104 may continue on to another destinationlocation if it is carrying multiple items that can be individuallydelivered to multiple destination locations.

At 404, the UAV 104 evaluates its remaining resources. In someembodiments, evaluating remaining resources includes evaluating anavailable range of the UAV, such as remaining battery power or fuellevels; evaluating the available sensors, such as a digital camera, anavailable memory, or a communication capability; or evaluating aremaining time to perform a delivery action within a delivery window, orto perform a surveillance action within a surveillance window. In someembodiments, evaluating remaining resources in operation 404 maycorrespond to the operation of determining resources in operation 304.

At 406, a surveillance action is determined based in part on theremaining resources of the UAV. A surveillance action may include adeviation from transit (such as a deviation from delivering a package orreturning to an origination location), gathering surveillance data (suchas imaging a surveillance location), and/or a return flight path (suchas returning to an origination location of the UAV). In someembodiments, determining the surveillance action includes selecting asurveillance action from one of many surveillance actions proposed by asurveillance system. In some embodiments, the surveillance action may bereceived as an interrupt. In some embodiments, determining thesurveillance action is based at least in part on the remaining resourcesof the UAV. As range is often a limiting factor in UAV travel,determining the surveillance action may include selecting a surveillanceaction that may be completed in the remaining available range of the UAV(i.e., without exhausting the resources of the UAV). In variousexamples, determining the surveillance action may be based on a timeconstraint (e.g., such as a time constraint to deliver a package,perform a surveillance action, return to an origin, or a subsequentdelivery and/or surveillance action), a priority of a surveillanceaction, the surveillance data to be gathered, environmental factorsincluding the weather, the physical proximity of a plurality ofsurveillance actions (i.e., the opportunity to complete multiplesurveillance actions), noise restrictions, flight path restrictions,surveillance restrictions, future expected surveillance actions, and/orfuture expected deliveries.

At 408, the surveillance action is executed to gather surveillance data.In various examples, surveillance data may be any data gathered by thesensors 904, such as such as digital camera(s) 906, spectral camera(s)908, audio sensor(s) 910, LIDAR/RADAR 912, GPS sensor(s) 914, chemicalsensor(s) 916, flight/delivery sensor(s) 918, or any combinationthereof.

At 410, surveillance data is transmitted. As described above, in someembodiments, operations 402-410 are performed by the UAV 104, whileoperations 412-426 are performed by the central controller 202.Therefore, in some embodiments, operation 410 includes transmittingsurveillance data from the UAV 104 to the central controller 202. Insome embodiments, the surveillance data is transmitted via interfaces934 of FIG. 9. For example, the surveillance data may be transmittedwirelessly over a network and directed to its destination, as would beunderstood by a person of ordinary skill in the art. The surveillancedata may be transmitted as compressed or uncompressed data, or may betransmitted with or without encryption. In some embodiments, thesurveillance data is transmitted in real time. In some embodiments, thesurveillance data may be transmitted to multiple recipients, such as thecentral controller 202, another UAV, a user device, or a serviceprovider. In some embodiments, geo-clipped data is transmitted (insteadof or in addition to surveillance data).

At 412, the surveillance data transmitted in operation 410 is received.In some embodiments, the surveillance data is received by centralcontroller 202. As one example, interfaces 220 of central controller 202receive surveillance data. As described above, the surveillance data maybe received as compressed or uncompressed data, or may be received withor without encryption. In an embodiment where operations 402-426 may beperformed in a single device, transmitting and receiving thesurveillance data may not be needed, and may be omitted. However, evenin an embodiment where operations 402-426 may be performed in a singledevice, surveillance data and/or geo-clipped data may be transmitted toor received by another device.

At 414, an analysis of the surveillance data may begin. Analysisoperations 416, 418, 420, 422, 424, and 426 may be performed in serial,in parallel, individually, or in any combination. Additionally,operations 416, 418, 420, 422, 424, and 426 may be repeated with respectto the surveillance data, and may be performed continuously on incomingstreams of surveillance data, for example. In some embodiments, theanalysis is performed automatically, such as via a machine visionalgorithm; in some embodiments, the analysis may be performed by a humananalyst; and in some embodiments, certain aspects of the analysis may beperformed automatically, and certain aspects may be performed by a humananalyst.

At 416, geo-clipped data is generated. The geo-clipped data may be thesame as or similar to the geo-clipped image 122 described in FIG. 1. Insome embodiments, the geo-clipped data may be surveillance data that hasbeen modified based on at least one location parameter. As discussedabove, the surveillance data may include data referring to one or moreexcluded locations 116 and surveillance location 118. In someembodiments, the surveillance location 118 is defined by the geo-fence120. In operation 416, the surveillance data is modified in order toexclude, blur, obscure, excise, mask, or hide data referring to theexcluded location 116. Generating geo-clipped data may includesurveillance data gathered using sensors 904. For example, geo-clippeddata may include data from digital camera(s) 906 and GPS sensor(s) 914.In this example, the data from GPS sensor(s) 914 may be used todetermine which portions of image data from digital camera(s) 906 referto the surveillance location 110, and which portions of the image datarefer to the excluded location 116. An example of generating geo-clippeddata can also be seen in FIGS. 5A, 5B, and 6.

At 418, the geo-clipped data is presented. In some embodiments,presenting geo-clipped data includes displaying a geo-clipped image on ascreen, for example. A non-exhaustive list of presenting geo-clippeddata includes presenting via an internet browser, an application, as adata file (such as a JPEG or MP4 file), in an email, as an attachment toan email, as a text message, as a hardcopy, and as portable media. Insome embodiments, the geo-clipped data may be presented on a display inreal time, while in other embodiments, the geo-clipped data may bepresented in a file repository accessible via an internet browser.

At 420, the analysis includes registering the surveillance data. In thisoperation, surveillance data of a surveillance location is stored in amemory. In some embodiments, the surveillance data relating to the samesurveillance location is gathered and stored to build a database ofregistered surveillance data. In some embodiments, registering thesurveillance data includes identifying objects of interest in asurveillance location. As a non-limiting example, an object of interestmay be the presence or absence of a physical feature, such as a car in adriveway at a surveillance location. In some embodiments, an object ofinterest may be identified automatically or manually, such as by a useror an operator. An example of identifying an object of interest isdescribed in connection with FIG. 8.

At 422, the surveillance data is compared. This operation includescomparing incoming surveillance data with the surveillance dataregistered in operation 420. In some embodiments, a machine visionprogram compares incoming surveillance data with the registeredsurveillance data to determine state changes between the registeredsurveillance data and the incoming surveillance data. In the exampledescribed above, surveillance data was registered, which includes theidentification of an object of interest, such as a car in a driveway. Asthe surveillance data is compared, a machine vision algorithm maydetermine a state change of the car in the driveway. Various examples ofthe state changes of the car in the driveway include whether the car ispresent, the location of the car in the driveway, whether the lights ofthe car are illuminated, or whether a window of the car is intact orbroken. In some embodiments, operations 420 and/or 422 include machinelearning and/or machine vision algorithms in its analysis.

At 424, the analysis includes determining the probability or confidencevalue of a surveillance event. In some embodiments, a surveillance eventis a disturbance or an important event warranting a surveillance alert.Some non-limiting examples of a surveillance event include an event suchas a property breech, an open door or garage door, a fire, or a brokenwindow. Events qualifying as a surveillance event may be set in advance,may be set automatically, or may be determined from predefined criteria.In some embodiments, a surveillance event may be manually indicated by auser. In some embodiments, it may be difficult to determine the state ofa surveillance event (e.g., whether an intruder is in a surveillancelocation). Therefore, in some embodiments, a probability or confidencevalue is determined that the perceived surveillance event is occurringor has occurred. For example, if a fire is directly observed insurveillance data, it may be determined with 100% probability that thesurveillance location is on fire. In another example, if a window on aproperty is broken, and an unknown vehicle is observed outside a house,the probability of an intruder in the house may be high, but possiblyless than 100% certain. In some embodiments, determining the probabilityor confidence value of a surveillance event may be based in part on thecomparison of surveillance data with registered surveillance data, asdescribed in operations 420 and 422. In other embodiments, determiningthe probability of a surveillance event is based in part on machinelearning, machine vision, and/or determined automatically or by a useror operator.

At 426, an alert is generated corresponding to a surveillance event.Non-limiting examples of an alert include alerts by telephone, email,text message, or through an alert application. An alert may be providedto multiple recipients, such as a service provider (such as a securityprovider), an operator, a user, or a municipal entity such a police orfire department. In some embodiments, alerts may be provided to a UAV,for example, as an interrupt or a surveillance action. In someembodiments, the alerts may depend on the type of surveillance event.For example, if the surveillance event is the determination that agarage door was left open, an alert may be a text message to a user,while if the surveillance event is a fire, an alert may be a textmessage or telephone call to a security provider or fire department. Insome embodiments, the generated alert may be based in part on whetherthe probability of the surveillance event is above a threshold. In someembodiments, the threshold may be set automatically or by a user. Insome embodiments, alerts may be automatically set or may be defined by auser. In some embodiments, an alert may be used to annotate ageo-clipped image to identify the surveillance event, such as with anarrow, a marker, or a color to draw a user's attention to thesurveillance alert. Surveillance alerts are discussed in more detail inconnection with FIG. 5B and FIG. 8.

FIG. 5A is a schematic diagram illustrating a surveillance image,processing of the surveillance image, and a post-processing surveillanceimage, in accordance with one embodiment disclosed herein.

A surveillance image 502 is an example of surveillance data generated bythe one or more sensors 904 of UAV 902. For example, the surveillanceimage 502 may be an image generated by digital camera(s) 906, spectralcamera(s) 908, LIDAR/RADAR 912, or any combination thereof. Thesurveillance image 502 may be a still image or may be a video.

In some embodiments, the surveillance image 502 includes within itsframe images of a surveillance location 504, a surveillance locationfence 506, and an excluded location 508. In some embodiments, thesurveillance location 504 may correspond with the surveillance location118; the surveillance location fence 506 may correspond with thegeo-fence 120; and the excluded location 508 may correspond with theexcluded location 116. In some embodiments, surveillance image 502 mayinclude public property (such as a street 510), or any property orobject beyond the surveillance location 504 or the excluded location508.

In some embodiments, the surveillance location fence 506 may correspondpartially with the geo-fence 120. In some embodiments, the surveillancelocation fence 506 may not completely define the boundaries of asurveillance region, and instead, a generated geo-fence may be larger orsmaller than the surveillance location fence 506. In some embodiments, ageo-fence may include the surveillance location as well as any publicland or private land open to the public. In some embodiments, datadefining a geo-fence may include one or more location parameters.

The surveillance image 502 may also include metadata 512 associated withthe surveillance image 502. In various examples, the metadata 512 refersto the conditions of the UAV at the time the surveillance image 502 wasgenerated, and may include GPS data or location data, altitude data ofthe UAV, direction and heading information of the UAV, and LIDAR/RADARmeasurements of the UAV to each object in the surveillance image 502.The metadata 512 may also include information relating to a particularsensor used to generate the surveillance image 502. For example, ifdigital camera(s) 906 was used to generate the surveillance image 502,the metadata 512 may also include information relating to camerasettings, such as focal length, aperture, shutter speed, metering mode,ISO speed, lighting conditions, timestamp, and/or image resolution. Insome embodiments, the metadata 512 may include inertial information ofthe UAV, such as accelerometer information in three dimensional space.In some embodiments, the metadata 512 may include information such asUAV identification information (e.g., a unique UAV identificationnumber), sensor information (e.g., make, model), environmentalinformation (e.g., weather), flight information, delivery actioninformation, and surveillance action information.

At 514, processing is performed to generate a geo-clipped image orvideo. In some embodiments, the processing in operation 514 maycorrespond to the processing in operation 416. In some embodiments, theprocessing in operation 514 may be performed by the UAV 104, the centralcontroller 202, or a combination thereof.

Processing to generate the geo-clipped image or video 514 includesreceiving the surveillance image 502, determining which portion of thesurveillance image 502 is to be geo-clipped, and generating ageo-clipped image or video. In some embodiments, determining whichportion of the surveillance image 502 to be geo-clipped is based in parton determining a geo-fence for the surveillance location 504. Forexample, the geo-fence for the surveillance location 504 may be based onthe surveillance location fence 506, or other information to establish avirtual barrier or boundary corresponding a real-world geographic area.In some embodiments, a geo-fence for surveillance location 504 isgenerated dynamically (e.g., such as in real time while processing 514is performed), and in some embodiments, a geo-fence for surveillancelocation 504 may be based on a predetermined set of boundaries.

In some embodiments, a machine vision algorithm analyzes thesurveillance image 502, the metadata 512, and a geo-fence associatedwith the surveillance location 504 to determine what portion of thesurveillance image 502 is to be geo-clipped. For example, a machinevision algorithm may use the metadata 512 to determine the location ofthe UAV relative to the geo-fence associated with the surveillancelocation 504, determine whether the surveillance image 502 includes thesurveillance location 504 or the excluded location 508, and determinewhat portion of the surveillance image 502 should be geo-clipped. Forexample, if the surveillance image 502 contains images of the excludedlocation 508, the surveillance data 502 may be modified to obscure theexcluded location 508. In some embodiments, the surveillance image 502may only contain images of the surveillance location 504, and no portionof the surveillance image 502 may need to be obscured. In someembodiments, a geo-clipped image can be generated automatically, by anoperator, or some combination thereof. For example, an operator mayverify that a machine vision algorithm has correctly geo-clipped thesurveillance image 502. Processing to gather or generate surveillancedata is discussed in more detail in connection with FIG. 6.

One example of a result of processing to generate a geo-clipped image orvideo is shown as geo-clipped surveillance image 516. As a non-limitingexample, the geo-clipped surveillance image 516 includes thesurveillance location 504 and the surveillance location fence 506. Insome embodiments, the geo-clipped surveillance image 516 includes ageo-clipped portion 518. The portion 518 is shown as covering theexcluded location 508. In some embodiments, the portion 518 may coverall portions of the image beyond the surveillance location 504, definedin part by the surveillance location fence 506. In some embodiments, thegeo-clipped portion 518 may only cover private property, such asexcluded location 508, and may not cover public property such as street510. In some embodiments, the geo-clipped portion is generated incompliance with all Federal, State, Municipal, and local laws, includingthose regarding rights of privacy and rights of publicity. As anon-limiting example, the geo-clipped portion 518 is shown as obscuredwith a pattern. However, in some embodiments, the geo-clipped portion518 may be represented as any color, with any level of translucency ordistortion that prevents view of specific objects or people in theunauthorized areas depicted in imagery. In some embodiments, thegeo-clipped portion 518 may be represented using pixelation, fogging,blur effects, posterization, censor bars, obscuring the area with pixelsof a constant color, removal of the area, cropping the area, orobscuring the area with random colors. Processing to generate ageo-clipped image or video is discussed in more detail in connectionwith FIG. 6.

After the processing 514 has been performed to generate the geo-clippedsurveillance image 516, the geo-clipped surveillance image 516 may bepresented. In some embodiments, presenting the geo-clipped surveillanceimage 516 may correspond to operation 418.

FIG. 5B is a schematic diagram illustrating a post-processingsurveillance image, surveillance events, and surveillance alerts, inaccordance with one embodiment disclosed herein. For simplicity, somenumbering shown in FIG. 5A has been omitted from FIG. 5B.

A geo-clipped surveillance image 520 shows an example of surveillanceevents 522, 524, and 526 occurring at surveillance location 504. Forexample, surveillance event 522 may correspond to a door of thesurveillance location 504 being open. Surveillance event 524 maycorrespond to a broken window at surveillance location 504. Surveillanceevent 526 may correspond to a person detected at the surveillancelocation 504. In some embodiments, the surveillance events 522 and 524may be detected using digital camera(s) 906 of the UAV 902. In someembodiments, the surveillance event 526 may be detected using spectralcamera(s) 908 of UAV 902, and may correspond to a thermal image of aperson detected behind a wall of the surveillance location 504. As shownin the geo-clipped surveillance image 520, sensor data may be layered toshow data corresponding to the digital camera(s) 906 and the spectralcamera(s) 908 in the same image.

Surveillance events 522, 524, and 526 may be identified in thesurveillance image 520 by surveillance alerts 528, 530, and 532. Forexample, the surveillance alerts 528, 530, and 532 may serve to identifythe location and probability or confidence value of the surveillanceevent occurring at the surveillance location 504. As discussed inconnection with FIG. 4, the sensors 904 may gather surveillance data ofthe surveillance location 504, and a machine vision algorithm and/orhuman analysis may process or review the surveillance data to determinethe probability or confidence value of a surveillance event, which isshown as an open door 528. Because the open state of the door may beobserved directly, the probability or confidence value of thissurveillance event may be high, and the surveillance image 520 may bemodified to identify the surveillance event 522 with the surveillancealert 528. The surveillance alert 528 corresponding to the surveillanceevent 522 may represent the location, severity, confidence value, and/orrisk associated with the surveillance event 522. For example, althoughthe surveillance alert 528 is shown as a patterned arrow in FIG. 5B, thesurveillance alert may be represented as a colored highlighting of thesurveillance image 520, or any other method to indicate the surveillanceevent.

As discussed above, surveillance event 526 may correspond to a persondetected at the surveillance location 504 via the spectral camera(s) 908of UAV 902. In some embodiments, the confidence value of thesurveillance event 532 may be higher or lower than the confidence valueof surveillance events 528 or 530, and the surveillance alert 532corresponding to surveillance event 526 may be represented accordingly.For example, if arrows are used to represent the surveillance alerts528, 530, and 532, the length, width, color, pattern, etc. of the arrowmay be modified to represent the type, confidence value, severity, levelof risk, immediacy, etc. of the surveillance event. In some embodiments,if a translucent color is used as the surveillance alert to indicate thesurveillance events, the color, size, level of translucency, etc. may bemodified to represent various factors of the surveillance event.

In some embodiments, the geo-clipped surveillance image 520 may bepresented to a user, service provider, security company, etc. in realtime, with a time delay, or in any suitable format. In some embodiments,the surveillance image 520 is an example of a surveillance image to bepresented in operations described elsewhere in this specification. Ifthe surveillance image 520 is presented to a user in a display or userinterface, in some embodiments, the user may select the surveillancealerts 528, 530, or 532 and may receive additional information regardingthe surveillance event. For example, if a user selects the surveillancealert 528, information about the surveillance event 522 may be provided,such as a description of the surveillance event 522, when thesurveillance event 522 was detected, or the probability or confidencevalue of the surveillance event 522. In some embodiments, the user maybe presented with an option to view surveillance data previouslyregistered in the central controller 202, for example, to view thesurveillance location 504 before the surveillance event 522 wasdetected. In some embodiments, the user may be presented with options toalert a service provider or a municipal entity, or the user may bepresented with a user interface to identify an object of interest or toadd or modify surveillance parameters for additional surveillance. Insome embodiments, for a surveillance event such as a broken window 524,selecting the surveillance alert 530 may bring up options available tothe user to remedy or fix the situation, such as advertisement for awindow repair service. In some embodiments, the user may be presentedwith a user interface similar to that shown in FIG. 8.

FIG. 6 is a flow diagram of an example process 600 for generatingsurveillance data.

In some embodiments, operations 602-622 may be performed by a UAV, suchas the UAV 104, during a surveillance action, while in some embodiments,the UAV 104 and the central controller 202 may perform operations602-622 in any combination, as would be understood by a person ofordinary skill in the art. Some of the operations in the process 600 maybe performed in serial, in parallel, individually, in any combination,or possibly in a different order than the order shown in FIG. 6.Additionally, operations 602-622 may be performed continuously during asurveillance action, for example.

At 602, a geo-fence is determined. The geo-fence may be the same as orsimilar to the geo-fence 120 described in FIG. 1, or the geo-fencedescribed in connection with FIGS. 5A and 5B. In some embodiments, thegeo-fence and/or geo-fence data is determined by the central controller202 in response to receiving a surveillance location from a user. Insome embodiments, the geo-fence is based on physical constraints of asurveillance location, and/or is based on portions of a surveillancelocation where a user has provided authentication or verification thatthey are an authorized party to perform surveillance (e.g., anauthorized area). For example, a geo-fence may provide a boundarybetween a surveillance location (such as surveillance location 504) andan adjacent location (such as excluded location 508, or an unauthorizedarea). In some embodiments, the geo-fence is determined dynamically(e.g., in real time) by the UAV during a surveillance action based oninformation received from sensors 904, such as GPS sensor(s) 914 andLIDAR/RADAR sensor(s) 912.

At 604, a determination is made whether an adjacent property is present.Operation 604 may include using sensors 904, such as digital camera(s)906, to view the surveillance location and to determine that no adjacentproperty (such as excluded location 116 or 508) is present. In someembodiments, a surveillance location may be a remote location where noexcluded locations are nearby, while in some embodiments, a surveillancelocation may be so large that no excluded locations may be observed. Insome embodiments, a surveillance location may be surrounded by publicland or property, private property open to the public, or a user mayhave received consent from owners of adjacent property. If no adjacentproperty is present (operation 604, “No”), surveillance data may begathered in operation 608, as described in connection with variousembodiments. If an adjacent property is determined to be present inoperation 604, processing continues to operation 606.

At 606, a determination is made whether it is possible to physicallyconstrain sensor(s). For example, if the digital camera(s) 906 are used,the digital camera(s) may be turned off to avoid the adjacent propertyuntil a time in which the UAV is in a location where surveillance datawould not be gathered of the adjacent property (i.e., a state of asensor may be changed). In some embodiments, the digital camera(s) 906may be zoomed in to only observe the surveillance location, or anaperture of the digital camera(s) 906 may be set and the digitalcamera(s) 906 may be focused such that the adjacent property may bephysically blurred, for example, using a technique such as bokeh. Insome embodiments, the UAV may change its direction, altitude, or heading(or otherwise reposition the camera) such that the adjacent propertywould not be in a field of view or within range or a particular sensor.In some embodiments, a flight path or deviation may be modified orplanned in such a manner as to avoid gathering surveillance data of theadjacent location (e.g., by approaching the surveillance location from aparticular direction). If it is determined that it is possible tophysically constrain the sensors without gathering surveillance data onthe adjacent property (operation 606, “Yes”), surveillance data may begathered at operation 608. If it is not possible to physically constrainthe sensors to avoid gathering surveillance data of the adjacentproperty, processing continues to operation 610.

At 608, surveillance data may be gathered. Because a determination ismade in 604 that an adjacent property is not present, or a determinationis made in 606 that the sensors may be physically constrained, thesurveillance data gathered in 608 may not include surveillance datacorresponding to an adjacent property or excluded location 508, and maynot need to be processed to generate geo-clipped data. In variousexamples, surveillance data may be any data gathered by the sensors 904,such as such as digital camera(s) 906, spectral camera(s) 908, audiosensor(s) 910, LIDAR/RADAR 912, GPS sensor(s) 914, chemical sensor(s)916, flight/delivery sensor(s) 918, or any combination thereof.

At 610, a determination is made whether to perform pre- orpost-processing to generate geo-clipped data. In some embodiments,pre-processing may be performed on the UAV, while in some embodiments,post-processing may be performed at the central controller 202, forexample. In some embodiments, the processing to generate geo-clippeddata may be within the processing capacity of the UAV, and accordingly adetermination may be made to perform pre-processing (operation 610,“Pre”), while in some embodiments, the processing to generategeo-clipped data may be beyond the processing capacity or capability ofa UAV, and accordingly, a determination may be made to performpost-processing (operation 610, “Post”). In some embodiments, thedetermination 610 to perform pre- or post-processing to generategeo-clipped data may depend on a type of adjacent property, a type ofsurveillance location, the available resources of the UAV, whethermanual (human) review of the surveillance data is required, the type ofsurveillance data to be gathered (e.g., using the digital camera(s) 906,spectral camera(s) 908, or audio sensor(s) 910), the priority of thesurveillance action, whether a surveillance event is detected, orwhether real time geo-clipped data is requested.

At 612, surveillance data is gathered. In some embodiments, operation612 may correspond to operations 308, 408, or 706 of FIGS. 3, 4, and 7,respectively. In various examples, surveillance data may be any datagathered by the sensors 904, such as such as digital camera(s) 906,spectral camera(s) 908, audio sensor(s) 910, LIDAR/RADAR 912, GPSsensor(s) 914, chemical sensor(s) 916, flight/delivery sensor(s) 918, orany combination thereof. As one non-limiting example, an example ofsurveillance data gathered in operation 612 may be the surveillanceimage 502 of FIG. 5. In the “post-processing” operation 612, gatheringsurveillance data may include surveillance data corresponding to anadjacent location, or an excluded location 508. In some embodiments,surveillance data gathered in operation 612 may not be presented to auser or service provider.

At 614, processing is performed to remove or obscure surveillance datato generate geo-clipped data. In some embodiments, operation 614 maycorrespond to operation 514 of FIG. 5 or operation 416 of FIG. 4, andthe result of the processing 614 may be similar to geo-clipped image516. In some embodiments, processing 614 may be performed by the UAV,while in some embodiments, the UAV may transmit the gatheredsurveillance data to the central controller 202, whereby the centralcontroller 202 may perform processing 614. Following operation 614, inoperation 620, the surveillance data or geo-clipped data is presented toa user or service provider.

Operations 612-614 and 616-620 may generate similar or the samegeo-clipped data, but may do so in different ways. For example, asdescribed above, post-processing to generate geo-clipped data includesremoving or obscuring surveillance data after the surveillance data isgathered. In contrast, pre-processing to generate geo-clipped dataincludes processing to discard data before unauthorized surveillancedata is stored in memory. In a sense, pre-processing operates bygeo-clipping the surveillance data before the surveillance data isgathered. In this way, privacy issues may be avoided because no data isstored of an adjacent property or excluded location 508. Operations616-620 are described in detail below.

At 616, using pre-processing, the operation includes receiving sensordata 616. In various examples, sensors may include sensors 904, such assuch as digital camera(s) 906, spectral camera(s) 908, audio sensor(s)910, LIDAR/RADAR 912, GPS sensor(s) 914, chemical sensor(s) 916,flight/delivery sensor(s) 918, or any combination thereof. At thispoint, sensor data may be stored temporarily (e.g., in volatile memory)or in a buffer.

At 618, processing determines whether the sensor data is within thegeo-fence determined in operation 602. For example, the UAV may usesensors 904 such as GPS sensor(s) 914 and/or LIDAR/RADAR 912 todetermine the UAV location relative to the geo-fence determined atoperation 602. Further, the UAV may use LIDAR/RADAR 912 to map out theUAV surroundings and to identify objects captured by digital camera(s)906 and/or spectral camera(s) 908. Based on the determined location ofthe UAV, the determined location of the geo-fence (including boundariesof the surveillance location, the excluded location, and any publicspaces), and the identification of objects in received sensor data 616,operation 618 may determine whether the sensor data corresponds tosurveillance data inside or outside of the geo-fence. In someembodiments, operation 618 is performed by a machine vision algorithm.

In some embodiments, the determination 618 whether sensor data is withina geo-fence may be based on a probability or confidence values, anddifferent regions of sensor data may include different confidencevalues, such as low, medium, and high. For example, a high confidencevalue may represent a strong likelihood the sensor data is inside thegeo-fence, and such data may be authorized data. A low confidence valuemay represent a strong likelihood that the sensor data is outside thegeo-fence, and the data may be unauthorized data. A medium confidencelevel may reflect data near a boundary of the geo-fence, and may or maynot be considered authorized data. In some examples, data labeled with amedium confidence level may be blurred or obscured in some manner beforestoring the data in memory. In some embodiments, operation 614 may useconfidence levels in a similar manner to remove or obscure surveillancedata, for example. In such an embodiment, the surveillance data may besegmented into a plurality of portions or regions, and a confidencevalue may be determined for each segmented portion or region. As wouldbe understood by a person of ordinary skill in the art, the terms“inside,” “outside,” “authorized,” and “unauthorized” are relativeterms, and may be used in accordance with the scope of the disclosure.For example, a surveillance location may be “outside” or “inside” ageo-fence, just as the area “inside” or “outside” of a geo-fence may bedesignated as an “authorized” or “unauthorized” area.

At 620, authorized data is written into memory. In some embodiments,memory may correspond to data store 926 of UAV 902, or data store 210 ofcentral controller 202. In some embodiments, unauthorized data isdiscarded, that is to say, unauthorized data is not stored in any memorybeyond a cache or a buffer. In this manner, privacy issues may beavoided because unauthorized data may never be stored in permanentmemory. As described above, data may be processed according to aconfidence level, and some authorized data may be partially obscured andstored in memory. In some embodiments, geo-clipped data generated byoperations 616-620 may be similar to the geo-clipped data generated byoperations 612-614, and may be similar to the geo-clipped image 122,516, or 520.

At 622, surveillance data or geo-clipped data may be presented. In someembodiments, operation 622 may correspond to operation 418 of FIG. 4.

In some embodiments, a combination operations described in connectionwith FIG. 6 (for example) may be used to gather surveillance data andgenerate geo-clipped data. As a non-limiting example, sensors may bephysically constrained by planning the flight path of the UAV (e.g., byplanning a surveillance action) to physically avoid an adjacent locationand to be activated at a particular time to avoid gathering data of anadjacent location. Next, pre-processing to generate geo-clipped data maybe used as a first pass to remove surveillance data where a confidencelevel is high that the data is unauthorized data. Further,post-processing may be used, either by a machine vision algorithm or bymanual (human) review, to confirm that the geo-clipped data containsonly authorized data.

FIG. 7A is a flow diagram of an example process 700 for processing aninterrupt to perform a surveillance action. In some embodiments,operations 702-710 may be performed by a UAV, such as the UAV 104. Insome embodiments, operations 702-710 may be performed by a surveillancesystem, such as the central controller 202, in combination with the UAV104, as would be understood by a person of ordinary skill in the art.Some of the operations in the process 700 may be performed in parallelor possibly in a different order than the order shown in FIG. 7A.

At 702, a surveillance interrupt is received. In some embodiments, thesurveillance interrupt is received by the UAV 104, while in someembodiments the surveillance interrupt is received by the centralcontroller 202. As part of receiving the surveillance interrupt, thesurveillance interrupt may be generated in various ways. For example,the surveillance interrupt may be generated based in part on asurveillance event, a surveillance alert, a request from a user orservice provider, a surveillance scheduling module, or another UAV. Insome embodiments, the surveillance interrupt may be generated by avoice-activated command device located at a surveillance location, whilein some embodiments, a surveillance interrupt may be generated by a userdevice (e.g., a “mayday” feature). In some embodiments, the surveillanceinterrupt may be generated by a traditional home surveillance system. Insome embodiments, the surveillance interrupt may be received byinterfaces 934 of UAV 902 and/or interfaces 220 of the centralcontroller 202.

At 704, a UAV may evaluate a priority of the surveillance interrupt andevaluate the UAV resources. In some embodiments, a surveillanceinterrupt may include a priority level, which indicates the urgency ofperforming the surveillance action. For example, if the UAV is en routeto deliver a package, the priority level may indicate that thesurveillance action should be performed before a delivery is to beperformed. In another example, the priority level may indicate that thesurveillance action should be performed after the delivery is to beperformed. In some embodiments, evaluating the priority level of thesurveillance interrupt is based in part on the resources of a UAV. Forexample, if there are sufficient resources (e.g., power, fuel, sensors,time, etc.) of a UAV such that the probability of performing both of thedelivery and the surveillance action is high, the UAV may determinewhich action to perform based on the most efficient use of resources.However, if there are not sufficient resources of the UAV, such that theprobability of performing both of the delivery and the surveillanceaction is low, the UAV may determine which action to perform based onthe priority level of the surveillance interrupt. In some embodiments,the probability of performing both the delivery and the surveillanceaction may be continuously determined until a decision must be made,based on the evaluated resources of the UAV and the most efficient useof the resources. In some embodiments, evaluating remaining resourcesmay correspond to the operation of determining resources in operation304 and/or operation 404.

At 706, surveillance is performed. In various examples, surveillancedata may be any data gathered by sensors 904, such as such as digitalcamera(s) 906, spectral camera(s) 908, audio sensor(s) 910, LIDAR/RADAR912, GPS sensor(s) 914, chemical sensor(s) 916, flight/deliverysensor(s) 918, or any combination thereof.

At 708, the resources of the UAV may be reevaluated. For example,because the surveillance action at 706 may have used UAV resources(e.g., power, fuel, memory, time, etc.), the resources of the UAV may bereevaluated. In some embodiments, reevaluating remaining resources maycorrespond to the operation of determining resources in operation 304and/or operation 404.

Based in part on the remaining resources reevaluated at 708, at 710, aflight path is determined. For example, if a delivery was postponed bythe surveillance interrupt, and a UAV has sufficient resources tocontinue with the delivery, a flight path is determined to continue withthe delivery. If the UAV does not have sufficient resources to performthe delivery, the UAV may return to an origination location withoutperforming the delivery.

FIG. 7B is a flow diagram of an example process 712 for selecting asurveillance action. In some embodiments, operations 714-718 may beperformed by a surveillance system, such as central controller 202. Insome embodiments, operations 720 and 722 may be performed by a UAV suchas UAV 104. In some embodiments, central controller 202 may performoperations 714-722, UAV 104 may perform operations 714-722, or UAV 104and central controller 202 may perform operations 714-722 in anycombination, as would be understood by a person of ordinary skill in theart. Some of the operations in the process 712 may be performed inparallel or possibly in a different order than the order shown in FIG.7B.

At 714, the central controller 202 may determine delivery requirements.For example, for each delivery, the central controller 202 may determinethe characteristics of a package (e.g., size, weight, or environmentalrequirements (such as temperature, humidity, etc.)), a destinationlocation, an origination location, and environmental factors such asweather, to determine the delivery requirements.

At 716, the central controller 202 may determine an anticipated unusedcapacity of a UAV. A UAV has finite resources, such as range, or anavailable amount of time in which a UAV may perform a delivery or returnfrom a delivery. The central controller 202 determines an anticipatedunused capacity based in part on the delivery requirements determined inoperation 714. An anticipated unused capacity may also be based in parton environmental factors such as weather, as well as the location ofvarious origins and destinations of a UAV.

At 718, one or more surveillance actions may be proposed within theanticipated unused capacity determined in operation 716. For eachsurveillance action to be proposed, a resource requirement, including apower requirement, a time requirement, or a sensor requirement, may bedetermined. In some embodiments, a plurality of surveillance actions maybe proposed for a plurality of surveillance locations. In such anembodiment, a non-limiting example may include a proposal for a firstsurveillance action at a first location and a second surveillance actionat a second location. In some embodiments, one or more surveillanceactions may be proposed for each of the plurality of surveillancelocations, with various levels of surveillance distinguishing thevarious surveillance actions. In such an embodiment, a non-limitingexample may include a proposal for a first surveillance action and asecond surveillance action to be performed at a first surveillancelocation, with the first and second surveillance actions varying in asurveillance depth. The surveillance depth may be based on the amount ofresources allocated for a surveillance action, such as UAV loiteringtime, surveillance altitude, surveillance speed, type(s) of sensorsused, surveillance data processing requested, and a priority of asurveillance action. In some embodiments, if operations 714-722 areperformed in a distributed manner, the results of any operations may becommunicated to any component of the system. For example, the centralcontroller 202 may transmit the proposed surveillance actions to the UAV104.

At 720, the UAV 104 may evaluate its remaining resources during transit.Operation 720 may be performed at any time, for example, before, duringor after a delivery of a package. In some embodiments, operation 720 ofevaluating the remaining resources during transit includes determining aremaining range of a UAV based on the current flight characteristics,environmental conditions, anticipated flight characteristics, and/orhistorical flight characteristics. In some embodiments, evaluatingremaining resources during transit 720 may correspond to the operationof determining resources in operation 304 and/or operation 404.

At 722, the UAV 104 may select one or more surveillance actions based onthe remaining resources evaluated in operation 720. The one or moresurveillance actions may be selected from the surveillance actionsproposed by the central controller 202 in operation 718. The selectionin operation 722 may be based in part on factors such as a priority of asurveillance action, a maximization of resources remaining after thesurveillance action, a maximization of surveillance actions to beperformed, a previously-performed surveillance action, or a surveillanceaction to be performed in the future. For example, the UAV 104 maycoordinate with other UAVs to maximize the number of surveillanceactions that may be performed by a fleet of UAVs. In some embodiments,surveillance actions may be selected dynamically and/or in real timebased in part on the location of individual UAVs of a fleet of UAVs, aswell as a surveillance action optimization algorithm.

FIG. 8 is an illustrative user interface 800 associated with determiningparameters of a surveillance system. The user interface 800 may beprovided on a user device such as a computer or mobile device. The userinterface 800 may be used to register a user, enter and modifysurveillance parameters, review surveillance data, and receive andmanage surveillance alerts. User interface 800 may be hosted in thesurveillance subscription module 216, a dedicated server, or a userdevice, as would be understood by a person of ordinary skill in the art.Furthermore, the user interface 800 may be configured in any manner, aswould be understood by a person of ordinary skill in the art.

The user interface 800 includes a surveillance tier selection 802. Thisselection allows a user to choose a desired surveillance tier, such aslow, medium, or high. In some non-limiting embodiments, selecting aradio button for a surveillance tier provides default settings for otherparameters. For example, a “Low” surveillance tier may include bydefault parameters including a low frequency (e.g., weekly), minimalmonitoring types (e.g., still images), and minimal alerts (e.g., emailonly). In other examples, a “Medium” surveillance tier may include bydefault parameters including a higher frequency (e.g., daily), a widerselection of monitoring types (e.g., still images and video), and awider selection of alerts (e.g., SMS/text messaging, video repository,and email). In other examples, a “High” surveillance tier provides themaximum amount of surveillance and options to a user. In someembodiments, selecting a surveillance tier removes surveillance optionsfrom the user interface 800, and in some embodiments, selecting asurveillance tier greys out surveillance options from the userinterface. In other examples, the selection of a surveillance tierdetermines the depth of surveillance, such the amount of resourcesallocated on a surveillance action, including UAV loitering time,surveillance altitude, surveillance speed, type(s) of sensors used,surveillance data processing requested, and priority. In other examples,pricing of the surveillance service is based in part on the selectedsurveillance tier.

Field 804 in the user interface 800 allows for a selection of asurveillance frequency. For example, the surveillance frequency may beselected such that surveillance actions are conducted weekly, daily,hourly, singly, or in some other determined fashion. In someembodiments, selecting a frequency of “single” allows a user to specifythe exact time, or a window of time, for the surveillance action to beperformed. In some embodiments, selecting a frequency of “other” allowsa user to specify irregular intervals, for example.

Field 806 in the user interface 800 allows for a selection of asurveillance monitoring type. Non-limiting examples in the field 806include “Still Images,” “Video,” “Infrared,” “Night,” “Audio,” “RealTime,” and “Other.” In some embodiments, the options for monitoring typemay depend on the types of available sensors in a UAV. The field 806 mayallow for the selection of multiple monitoring types.

Field 808 allows for a surveillance location to be specified. Forexample, an address of a surveillance location could be input into field808. In some embodiments, the address of a surveillance location maycorrespond to the address or location of a GPS-enabled device, such as asmartphone or a car. In some embodiments, a user must submit proof ofownership or control of the surveillance location receivingsurveillance, and possibly consent of other people residing at thatlocation. When the user is verified as owning or controlling theproperty, they may be considered an authorized party. If a user cannotverify they are an authorized party, the surveillance location may berejected and/or a surveillance location may not be performed at thatsurveillance location.

In response to inputting an address in the field 808, field 810 mayprovide an image of a surveillance location corresponding to the addressin the field 808. In some embodiments, the image provided in the field810 may include a satellite photo of the surveillance location. In someembodiments, the image provided in the field 810 may include an abstractrepresentation of the surveillance location. In some embodiments, theimage provided in the field 810 may include metadata about the objectsin the image, such as an address, a street name, and/or a proposedgeo-fence.

The field 810 may include a function to specify objects of interest. Insome embodiments, the field 810 displays a surveillance location house814, a surveillance location garage 812, and an excluded location house816. In some embodiments, the field 810 may provide an indication of thegeo-fence associated with the surveillance location house 814. In someembodiments, the field 810 allows for a selection of an object ofinterest. Such a selection may include using a mouse, a pointer, orother graphical user interface input (e.g., a finger or stylus, in thecase of a touch screen) to select a region or object within the field810. By identifying a region or an object as an object of interest, asurveillance action associated with this surveillance locationprioritize resources to gathering surveillance data corresponding to theidentified region or object, such as planning a surveillance action toprovide flight paths or perspectives conducive for imaging the region orobject. In some embodiments, if a region or an object is identified thatis outside of the surveillance location (or geo-fence, if available) asan object of interest, the user interface 800 may prohibit that regionor object from being selected. In other embodiments, if an object ofinterest is determined to be outside the surveillance location, arequest may be made (e.g., by a user to the central controller 202) toreevaluate the boundaries of the surveillance location. In someembodiments, a user must submit proof of ownership or control of thesurveillance location receiving surveillance, and possibly consent ofother people residing at that location. In some embodiments, a user mustprovide authentication that he or she is associated with thesurveillance location, such as by providing a lease, a utility bill, orother property records. In some embodiments, authentication may beprovided via a billing address or a shipping address associated with auser account. In some embodiments, an object of interest may bespecified after surveillance data has been gathered. For example, a usermay receive geo-clipped surveillance data of a surveillance location andmark or specify an object of interest within the geo-clippedsurveillance data.

In field 818 of user interface 800, alerts may be specified for thesurveillance system. Non-limiting examples of alerts include SMS (ShortMessage Service)/Text Messaging, Video Repository, Email, Telephone,Service Provider, Police/Fire, and Other. In some embodiments, one ormore alerts may be specified. When a surveillance event has beendetected (or the probability of the surveillance event is above athreshold probability), an alert may be generated. A process fordetermining surveillance events and generating alerts was discussedabove in connection with processes 424 and 426 of FIG. 4. In someembodiments, the surveillance data provided in an alert may begeo-clipped surveillance data (such as still images or video), and insome embodiments, the surveillance data may be analyzed and/or edited bya machine vision algorithm to present information relating to orindicating a surveillance event.

For a SMS/Text Message alert in the field 818, an alert may be providedvia text messaging. As non-limiting examples, such an alert may includegeo-clipped images and/or videos, a textual description of asurveillance event, audio information, or a hyperlink to a webpageproviding more information about a surveillance event.

For a Video Repository alert in the field 818, surveillance data may bestored in a data repository and provided to a user via a webpage. Insome embodiments, geo-clipped images and/or videos may be provided viathe Video Repository. As non-limiting examples, the Video Repository maybe provided by the central controller 202, and accessed as a webpageusing an internet browser, or may be provided as a surveillanceapplication.

For an Email alert in the field 818, an email may be provided to one ormore email addresses. As non-limiting examples, such an alert mayinclude geo-clipped images and/or videos, a textual description of asurveillance event, audio information, or a hyperlink to a webpageproviding more information about a surveillance event.

For a Telephone alert in the field 818, a telephone call may be providedby an automated caller or an operator indicating a surveillance event.

For a Service Provider alert in the field 818, a user may wish toprovide alerts to a service provider, such as a security company. Forexample, the system provided by the central controller 202 may beprovided in addition to or in conjunction with a security company. If asurveillance event is detected, a service provider may be alerted, forexample, to dispatch surveillance personnel to verify or investigate asurveillance event.

A Police/Fire alert may be generated for certain events, such anaccident, break-in, or fire. In some embodiments, a Police/Fire alertmay not be selected in user interface 800, but may be invoked undercertain default circumstances.

Additionally, the field 818 includes an option for an “Other” alert.Here, the alert may be specified by the user.

In field 820 of user interface 800, information may be provided relatingto the authorization of the surveillance location. In some embodiments,a user must submit proof of ownership or control of the surveillancelocation receiving surveillance, and possibly consent of other peopleresiding at that location. When the user is verified as owning orcontrolling the property, they may be considered an authorized party. Ifa user cannot verify they are an authorized party, the surveillancelocation may be rejected and/or a surveillance location may not beperformed at that surveillance location. In another example, while alandlord may own a property, the landlord may not live at the property,and may not be considered an authorized party if the tenants do notconsent to any surveillance action. Field 820 may display the status ofan authorization for a surveillance location or a surveillance action.In some embodiments, field 820 may include fields to upload informationto provide documentation that a user is an authorized party (such as alease or property records), or may include fields to edit informationrelating to a user's authorization.

FIG. 9 illustrates an example UAV 902 that is supported by the centralcontroller 202. In various examples, the UAV 902 can correspond to theUAV 104. The UAV 902 may be equipped with sensors 904 that performsurveillance actions, and monitor the operation and functionality of thephysical structures and the physical systems of the UAV 902. In someembodiments, the sensors 904 gather surveillance data during asurveillance action. The sensors 904 can include, but are not limitedto, digital camera(s) 906, spectral camera(s) 908, audio sensor(s) 910,LIDAR/RADAR 912, global positioning system (GPS) sensor(s) 914, chemicalsensor(s) 916, and flight/delivery sensor(s) 918.

In various embodiments, the digital camera(s) 906 can be used to provideimaging for the UAV 902 during flight and/or during a surveillanceaction. For example, the digital camera(s) 906 can be used to providereal time still images or real time video of a surveillance location. Insome embodiments, the digital camera(s) 906 may include stereoscopiccameras with varying focal lengths to provide three dimensional images.For example, when viewing a stereoscopic image produced by the digitalcamera(s) 906, the portions of an image closer to the digital camera(s)906 may be in focus, while the portions of the image further away fromthe digital camera(s) 906 may be blurry. In some embodiments, thedigital camera(s) 906 may be used for machine vision, navigation, etc.

In some embodiments, the spectral camera(s) 908 may provide infraredimaging, near-infrared imaging, thermal imaging, and/or night visionimaging. In some embodiments, the spectral camera(s) 908 may providestill images and/or video imaging capabilities. In some embodiments, thespectral camera(s) 908 and/or the digital camera(s) 906 can be usedtogether to provide multi-dimensional (and/or multi-layered)surveillance images representing a variety of light spectrums. Forexample, a surveillance action may use the digital camera(s) 906 toidentify a broken window at a surveillance location, and the spectralcamera(s) 908 may be used to identify a person inside of a building,while combining the data into a multi-dimensional or multi-layeredimage. In some embodiments, the spectral camera(s) 908 can be used toprovide a thermal image of a building, for example, to determine theenergy efficiency of the building.

In some embodiments, the audio sensor(s) 910 can be used to detect noiseat a surveillance location. The audio sensor(s) 910 may include filtersand/or audio processing to compensate for noise generated by the UAV902.

In various examples, the LIDAR/RADAR 912 (laser illuminated detectionand ranging/radio detection and ranging) may provide detection,identification, and precision measurement of a distance to asurveillance target. For example, the LIDAR/RADAR 912 may provideaccurate mapping of a surveillance location, and/or determination of thelocation of an object of interest. In some embodiments, the LIDAR/RADAR912 may be used in part to determine the location of the UAV 902relative to a geo-fence, such as the geo-fence 120. In variousembodiments, the LIDAR/RADAR 912 may be used to provide navigation ofthe UAV 902, in conjunction with other of the sensors 904.

In some embodiments, the global positioning system (GPS) sensor(s) 914may provide location and time information to the UAV 902. For example,the GPS sensor(s) 914 may provide metadata to the digital camera(s) 906and the spectral camera(s) 908 as the location of the UAV when an imageis generated. An example of such metadata may be the metadata 512 inFIG. 5A. In some embodiments, the GPS sensor(s) 914 may be used ingenerating geo-clipped surveillance data, such as a geo-clipped image orvideo.

In some embodiments, the chemical sensor(s) 916 can be used to measurethe presence of various chemicals in the air. For example, the chemicalsensor(s) 916 can be used to detect chemicals to determine the presencefire, or may be used to detect a chemical leak.

In some embodiments, the flight/delivery sensor(s) 918 may includeaccelerometer(s), gyroscope(s), proximity sensor(s), temperaturesensor(s), moisture sensor(s), voltage sensor(s), current sensor(s), andstrain gauge(s). In some embodiments, the flight/delivery sensor(s) 918may provide support to the UAV 902 physical systems. In someembodiments, data from the flight/delivery sensor(s) 918 may be used inconjunction with surveillance data, for example, in generatinggeo-clipped surveillance data.

In some embodiments, the UAV 902 can include one or more processor(s)920 operably connected to computer-readable media 922. The UAV 902 canalso include one or more interfaces 934 to enable communication betweenthe UAV 902 and other networked devices, such as the central controller202, a surveillance location, a service provider, a user device, orother UAVs. The one or more interfaces 934 can include network interfacecontrollers (NICs), I/O interfaces, or other types of transceiverdevices to send and receive communications over a network. Forsimplicity, other computers are omitted from the illustrated UAV 902.

The computer-readable media 922 may include volatile memory (such asRAM), non-volatile memory, and/or non-removable memory, implemented inany method or technology for storage of information, such ascomputer-readable instructions, data structures, program modules, orother data. Some examples of storage media that may be included in thecomputer-readable media include, but are not limited to, random accessmemory (RAM), read only memory (ROM), electrically erasable programmableread only memory (EEPROM), flash memory or other memory technology,compact disk (CD-ROM), digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can be accessed by a computingdevice.

In some embodiments, the processor(s) 920 and the computer readablemedia 922 can correspond to the processor(s) 204 and computer-readablemedia 206 associated with the central controller 202.

In some embodiments, the computer-readable media 922 can include anoperating system 924 and a data store 926. The data store 926 may beused to locally store sensor data that corresponds to the sensor 904data. As non-limiting examples, the data store 926 may storesurveillance data, data relating to delivery actions and surveillanceactions, and scheduling information.

In various examples, the computer-readable media 922 can include aresource management module 928. The resource management module 928 canmonitor the resources of the UAV 902. In some embodiments, resources tobe monitored include total resources, used resources, and availableresources. Resource management module 928 may also monitor historicalresource usage and compare predicted resources with resources actuallyused. In some embodiments, resources may include a power resource (e.g.,battery power or fuel levels indicating the range of the UAV), a sensorresource indicating the available sensors of a UAV (e.g., a digitalcamera, an available memory available to store surveillance data, or acommunication capability), or a time resource indicating a timeconstraint (e.g., a time constraint to deliver a package, perform asurveillance action, return to an origin, or a subsequent deliveryand/or surveillance action). In some embodiments, the resourcemanagement module 928 may be configured to determine the resourcesrequired by a delivery action, and/or determine the resources requiredby a surveillance action. In some embodiments, the determination isbased in part on environmental factors such as the weather, includingwind speed, direction, a weather forecast, temperature, time, ambientlight, etc. In some embodiments, determining the resources may alsoinclude determining a priority of a resource, a delivery action, or asurveillance action. In some embodiments, the resource management module928 may perform operations of evaluating remaining resources asdescribed in operation 304 and operation 404, or example.

In various examples, the computer-readable media 922 can include ascheduling module 930. The scheduling module 930 can provide schedulingof a delivery action and a surveillance action of a UAV. In someembodiments, the UAV may receive one or more delivery actions and one ormore surveillance actions from the central controller 202. In someembodiments, the scheduling module 930 includes a priority of a deliveryaction and/or a priority of a surveillance action. In some embodiments,the surveillance module 930 receives a surveillance interrupt anddetermines a schedule based on the priority of a delivery action and/ora surveillance action.

In various embodiments, the scheduling module 930 can correspond to thedelivery scheduling module 212 and/or surveillance scheduling module 214of the central controller 202. The functionality of the schedulingmodule 930 is substantially identical to the delivery scheduling module212 and/or the surveillance scheduling module 214. In some embodiments,the scheduling module 930 receives scheduling information from thedelivery scheduling module 212 and/or the surveillance scheduling module214. In other embodiments, the scheduling module 930, the deliveryscheduling module 212, and/or the surveillance scheduling module 214 mayoperate simultaneously. In this instance, the processing derived byeither one of the scheduling module 930, the delivery scheduling module212, and/or the surveillance scheduling module 214 can be used to checkthe processing determined by the other. In various examples, thescheduling of deliveries and/or surveillance actions solely by theeither the scheduling module 930, the delivery scheduling module 212, orthe surveillance scheduling module 214, or in any combination. In someembodiments, the scheduling module 930 may perform all schedulingprocesses described herein, including the scheduling processes describedin connection with the figures of this disclosure.

In various examples, the computer-readable media 922 can include asurveillance module 932. In some embodiments, the surveillance module932 may control the sensors 904 of the UAV 902 to perform surveillanceactions. In some embodiments, the surveillance module 932 receivessensor data from the sensors 904 as surveillance data and may modify thesurveillance data to generate geo-clipped surveillance data. In someembodiments, the surveillance module 932 includes a machine visionalgorithm that registers surveillance data, compares surveillance data,determines the probability of a surveillance event, and generates one ormore alerts of the surveillance event. In some embodiments, thesurveillance module 932 can include processing as described connectionwith the figures of this disclosure.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as illustrative forms ofimplementing the claims.

The invention claimed is:
 1. A method comprising: determining, based atleast in part on a geo-fence corresponding with a boundary surrounding asurveillance location, geo-fence data corresponding to the surveillancelocation, the geo-fence data distinguishing between an authorized areaand an unauthorized area; receiving a surveillance image captured by acamera of an unmanned aerial vehicle (UAV), the surveillance imagedepicting the surveillance location and including first image data ofthe authorized area within the geo-fence and second image data of theunauthorized area outside the geo-fence; receiving surveillance imagemetadata associated with the surveillance image, the surveillance imagemetadata including at least location data of the UAV and a directionthat the camera faces during a capture of the surveillance image;analyzing, by the UAV, the surveillance image using the geo-fence dataand the surveillance image metadata to identify the first image data ofthe authorized area and the second image data of the unauthorized area;generating a geo-clipped surveillance image by modifying thesurveillance image to at least obscure or remove the second image datadepicting the unauthorized area while maintaining the first image dataof the authorized area; detecting, based at least in part on the firstimage data, an event; determining, based at least in part on apredefined criterion, that the event qualifies as a surveillance event;determining a confidence value associated with the surveillance eventbased in part on the analyzing the surveillance image captured by theUAV; generating a surveillance alert for the surveillance event based atleast in part on the confidence value being above a threshold; andadding a visual indication of the surveillance alert to the geo-clippedsurveillance image.
 2. The method of claim 1, wherein the adding thevisual indication of the surveillance alert includes at least one of:changing a color of a portion of the geo-clipped surveillance image tovisually communicate the surveillance alert, or adding a descriptivemarker to the surveillance image as the visual indication.
 3. The methodof claim 1, wherein the geo-fence data further designates publiclyaccessible land that is permitted to be image-captured, and wherein thegeo-clipped surveillance image further depicts at least a portion of thepublicly accessible land.
 4. The method of claim 1, wherein thesurveillance image metadata further includes at least one of: analtitude of the UAV or the camera; a heading of the UAV; or LIDAR/RADARdata from the UAV.
 5. The method of claim 1, wherein the determining thegeo-fence data further includes comparing global positioning system(GPS) coordinates to property boundaries to create the geo-fence todefine the authorized area that correspond with the property boundaries.6. A system comprising: one or more processors; and memory coupled tothe one or more processors, the memory including one or moreinstructions that when executed by the one or more processors, cause theone or more processors to perform acts comprising: determining, based atleast in part on a geo-fence corresponding with a boundary surrounding asurveillance location, geo-fence data corresponding to the surveillancelocation, the geo-fence data distinguishing between an authorized areaand an unauthorized area; receiving a surveillance image captured by acamera of an unmanned aerial vehicle (UAV), the surveillance imagedepicting the surveillance location and including first image data ofthe authorized area within the geo-fence and second image data of theunauthorized area outside the geo-fence; receiving surveillance imagemetadata associated with the surveillance image, the surveillance imagemetadata including at least location data of the UAV and a directionthat the camera faces during a capture of the surveillance image;analyzing, by the UAV, the surveillance image using the geo-fence dataand the surveillance image metadata to identify the first image data ofthe authorized area and the second image data of the unauthorized area;generating a geo-clipped surveillance image by modifying thesurveillance image to at least obscure or remove the second image datadepicting the unauthorized area while maintaining the first image dataof the authorized area; detecting, based at least in part on the firstimage data, an event; determining, based at least in part on apredefined criterion, that the event qualifies as a surveillance event;determining a confidence value associated with the surveillance eventbased in part on the analyzing the surveillance image captured by theUAV; generating a surveillance alert for the surveillance event based atleast in part on the confidence value being above a threshold; andadding a visual indication of the surveillance alert to the geo-clippedsurveillance image.
 7. The system of claim 6, wherein the visualindication of the surveillance alert includes at least one of: changinga color of a portion of the geo-clipped surveillance image to visuallycommunicate the surveillance alert; or adding a descriptive marker tothe surveillance image as the visual indication.
 8. The system of claim6, wherein the surveillance image metadata further includes at least oneof: an altitude of the UAV; an altitude of the camera; a heading of theUAV; LIDAR data from the UAV; or RADAR data from the UAV.
 9. The systemof claim 6, wherein the adding the visual indication of the surveillancealert is based in part on the confidence value.
 10. The system of claim6, wherein the acts further comprise causing the camera of the UAV tocapture the surveillance image.
 11. The system of claim 6, wherein thedetermining the geo-fence data corresponding to the surveillancelocation includes identifying a public land that is authorized for imagecapture, and wherein the surveillance image includes imagery of at leastpart of the public land.
 12. The system of claim 6, wherein thedetermining the geo-fence data further includes comparing globalpositioning system (GPS) coordinates to property boundaries to createthe geo-fence to define the authorized area that correspond with theproperty boundaries.
 13. The system of claim 6, wherein the surveillancealert is sent to a computing device associated with an owner of thesurveillance location.
 14. A non-transitory computer-readable mediumstoring instructions executable by one or more processors, wherein theinstructions, when executed, cause the one or more processors to performoperations comprising: determining, based at least in part on ageo-fence corresponding with a boundary surrounding a surveillancelocation, geo-fence data corresponding to the surveillance location, thegeo-fence data distinguishing between an authorized area and anunauthorized area; receiving a surveillance image captured by a cameraof an unmanned aerial vehicle (UAV), the surveillance image depictingthe surveillance location and including first image data of theauthorized area within the geo-fence and second image data of theunauthorized area outside the geo-fence; receiving surveillance imagemetadata associated with the surveillance image, the surveillance imagemetadata including at least location data of the UAV and a directionthat the camera faces during a capture of the surveillance image;analyzing, by the UAV, the surveillance image using the geo-fence dataand the surveillance image metadata to identify the first image data ofthe authorized area and the second image data of the unauthorized area;generating a geo-clipped surveillance image by modifying thesurveillance image to at least obscure or remove the second image datadepicting the unauthorized area while maintaining the first image dataof the authorized area; detecting, based at least in part on the firstimage data, an event; determining, based at least in part on apredefined criterion, that the event qualifies as a surveillance event;determining a confidence value associated with the surveillance eventbased in part on the analyzing the surveillance image captured by theUAV; generating a surveillance alert for the surveillance event based atleast in part on the confidence value being above a threshold; andadding a visual indication of the surveillance alert to the geo-clippedsurveillance image.
 15. The non-transitory computer-readable medium ofclaim 14, wherein the visual indication of the surveillance alertincludes at least one of: changing a color of a portion of thegeo-clipped surveillance image to visually communicate the surveillancealert; or adding a descriptive marker to the surveillance image as thevisual indication.
 16. The non-transitory computer-readable medium ofclaim 14, wherein determining the geo-fence data corresponding to thesurveillance location includes identifying a public land that isauthorized for image capture, and wherein the surveillance imageincludes imagery of at least part of the public land.
 17. Thenon-transitory computer-readable medium of claim 14, wherein theoperations further comprise causing the camera of the UAV to capture thesurveillance image.
 18. The non-transitory computer-readable medium ofclaim 14, wherein the determining the geo-fence data corresponding tothe surveillance location includes identifying a public land that isauthorized for image capture, and wherein the surveillance imageincludes imagery of at least part of the public land.
 19. Thenon-transitory computer-readable medium of claim 14, wherein thedetermining the geo-fence data further includes comparing globalpositioning system (GPS) coordinates to property boundaries to createthe geo-fence to define the authorized area that correspond with theproperty boundaries.
 20. The non-transitory computer-readable medium ofclaim 14, wherein the surveillance alert is sent to a computing deviceassociated with an owner of the surveillance location.